Vision as Dance? Three Challenges for Sensorimotor Contingency Theory*

In Action in Perception Alva Noë develops and presents a sensorimotor account of vision and of visual consciousness. According to such an account seeing (and indeed perceiving more generally) is analysed as a kind of skilful bodily activity. Such a view is consistent with the emerging emphasis, in both philosophy and cognitive science, on the critical role of embodiment in the construction of intelligent agency. I shall argue, however, that the full sensorimotor model faces three important challenges. The first is to negotiate a path between two prima facie unsatisfactory readings of the central claim that conscious perceptual experience is constituted by knowledge of patterns of sensorimotor dependence. The second is to convince us that the sensorimotor contribution, in such cases, is actually constitutive of perceptual experience rather than merely causally implicated in the origination of such experience. And the third is to respond to the important challenge raised by what I will dub 'sensorimotor summarizing' models of the relation between conscious experience and richly detailed sensorimotor routines. According to such models conscious perceptual experience only rather indirectly reflects the rich detail of our actual sensorimotor engagements, which are instead lightly sampled as a coarse guide, optimized for planning and reasoning, and geared and filtered according to current needs and purposes. 1. The Central Claim Dance is, without doubt, a kind of skilful bodily activity. Subtract the body and its activity and – modulo only the severest forms of contemporary exploration – the dance itself must disappear from view. The central claim of Alva Noë's important, stylish and challenging treatment is that perception is more like dance than philosophers and cognitive scientists have (mostly iii ) noticed. For like dance "perceiving is a kind of skilful bodily activity" (Noë (2004) p.2). What can this mean? It does not mean, merely, that you need a body (or at least, some sense organs and a brain) to perceive. Rather, it means that skilful bodily action and perception are intimately entangled. The key to this shared intimacy is the idea that conscious perceptual experience consists in a perceiver's implicit knowledge of 'sensorimotor contingencies': rules or regularities relating sensory inputs to movement, changes and action. Implicit knowledge of such contingencies typically amounts to having a set of expectations concerning the "way sensory stimulation varies as a result of movement" (75). Both the character (the 'what it is like' of vision, touch, hearing etc) and the contents (concerning space, color, shape etc) of our perceptual experiences are said to be determined by our implicit knowledge of SMC's (sensorimotor contingencies), or (as Noë now prefers to say) by our 'sensorimotor expectations'). To illustrate this, consider a visually presented horizontal line and a looming ball heading at alarming velocity towards your face. These two distinct experiences correspond, on Noë's account, to two distinct signatures in sensorimotor space. Thus, if you move your eyes along the straight line, the retinal stimulation is invariant, whereas if you move your eyes up or down relative to the line, there is a sudden shift. A stationary ball would display a very different profile to this, while the moving ball presents a distinctive looming pattern that can (and probably should) be terminated by an act of ducking. The first claim, as I shall understand it, is that differences in what we perceptually experience correspond to differences in the sensorimotor signature associated with certain objects, properties and states of affairs. If two things look different, they do so because in encountering them we bring to bear (rightly or wrongly) different sets of sensorimotor expectations. But despite such differences, for all visually presented objects there will be some large parts of the sensorimotor signatures in common. It is these commonalities that make the experiences visual (rather than, say, auditory). For example, vision (unlike audition or touch) samples the front or facing sides of objects, and so on. The visual attributes of sensed objects are thus that subset of the signature sensorimotor contingencies that pertain to the distinctive ways that the visual sense can sample the real properties of objects. Thus, the very same real property (e.g. size) may be apprehended by vision or sometimes (for small objects) by touch. But the mode of sampling varies dramatically, and with it the associated sensorimotor contingencies. The visual properties of an object are, on this account, nothing but certain objective properties (e.g. size, shape and color) sampled in a distinctive way. Of course, some properties (such as color) are available to humans only via one modality (vision). But we can imagine e.g. color judgments made by a prosthetic sensor and reported via audible tones. What makes our normal experience of color visual thus remains the details of a certain mode of sampling. The central idea is thus that the contents and character of perceptual experience are determined by implicit knowledge of various types of what I am calling 'sensorimotor signature'. These signatures can include various elements, some having to do with the expected effects of our own movement on the input, others concerning the way changes to external conditions (lighting etc in the case of color) will affect the input. Some of the elements that enter into sensorimotor signatures thus turn on objective features of the world, while others turn on idiosyncratic features of our own sensory apparatus (the curve of the eyeball, the spacing of photoreceptive cells, etc). Uniting all the cases is the guiding idea that perceived content and character depends on expectancies concerning the future unfolding, under various conditions, of patterns of sensory stimulation. The first challenge can now be stated. What does it mean to speak of expectancies concerning the future unfolding of patterns of sensory stimulation? In particular, are we to understand 'expectancies concerning sensory stimulation' as a personal or sub-personal phenomenon? It would be a personal level phenomenon if our expectancies concerned sensory experiences themselves (e.g. we expect the ball to (in one sense) look bigger and bigger as it approaches our head). It would be a sub-personal phenomenon if what was at issue was some neural network's being able to predict the increasing area of some pattern of sensory stimulation defined at, say, the retina. Both readings face difficulties. The first reading, as Noë (87, 228) notes, courts circularity. It assumes we already have perceptual experiences of object appearances (e.g. the way the plate looks elliptical from an angle-see p.84) and then builds further kinds of content (e.g. the plate's also looking round) from our knowledge about how e.g. that elliptical look would vary as a result of movement around the plate. The experience of roundness just is, on this account, the active deployment of our implicit understanding of how the various looks would alter as a result of motion. Put like that, the story assumes 'ways things look visually' and does not explain them in any fundamental sense. Instead, what is explained is our grasp of a certain kind of visual content (roundness) on the basis of other kinds of visual content (regular variations in elliptical looks, etc). To avoid the threat of circularity, Noë suggests we should understand what it is to experience a look as nothing but our drawing on a certain set of more basic sensorimotor skills. The elliptical look of the plate is cashed out in terms of my ability to move my hand in a certain manner were I to try to indicate the shape as it appears in my visual field. Noë writes that 'In this way, my sensorimotor skill is drawn on to constitute my experience of the shape' (89). Looks are thus relations between the sensorimotor repertoire of embodied agents and objects. The virtue of this proposal is that it avoids a phenomenal (and hence circular) model of looks while (it seems) keeping the story operating at the personal level i.e. at the level of our actual understanding of our own sensorimotor space. The obvious drawback with this proposal is that it leaves it unexplained why knowledge concerning the relevant sensorimotor space should result in the experience of anything at all. Perhaps this gap can be filled (see Clark (2001) and Pettit (2003) for some attempts). But nothing in Noë's account appears apt to plug the gap. A second reading, more in line with the earlier account laid out by O'Regan and Noë (2001) would pitch the whole story at the sub-personal level. What is doing the work, on this reading, is knowing (non-consciously, implicitly) how the energy patterns impacting your sense organs will vary in response to your own actions and movements. (This is, I note, just the sort of knowledge that would be acquired by a simple recurrent neural network trained to predict the next sensory input from information concerning the present sensory state plus efferent copy of a motor command). This reading also avoids circularity (and hence was preferred in the earlier worksee Noë p.228) but seems to have been dropped in Noë's book because of a lack of 'phenomenological aptness' (228). The worry seems to be that unconscious knowledge (of sensorimotor contingencies) provides no obvious basis for phenomenal consciousness, which is what Noë seeks to explain. But as we just saw, neither does the version that invokes actual experiences of looks. By offering an account apparently pitched at the personal level that nonetheless denies that it is trading in phenomenality as such (but only in our own grasp of our sensorimotor relations to objects), Noë may be hoping to somehow find a safe haven between the two readings. But I am not convinced that any stable intermediate line is actually displayed in the text. For either our own grasp of our sensorimotor relations to objects is something that figures in our experience or it is not. If it is, then the central claim looks circular (as an account of visual experience). If it is not, then we lose our grip on 'phenomenological aptness' and a gap looms between possessing these bodies of skill and actual visual experience. What is lacking is a persuasive account of why it is that certain patterns of sensorimotor knowing (understood in a staunchly non-experiential way) should make it the case that a creature has some form of perceptual experience. My own view is that as it stands, the account on offer is best viewed as a proposal concerning how conscious perceptual experiences get their contents, rather than an account of how there come to be conscious perceptual experiences at all. Considered in this light, we can perhaps say that experiences of looks get their content from our basic repertoires of sensorimotor skills (or orienting, grasping, pointing etc) while other experiences (e.g. seeing the top of the cup as circular rather than elliptical etc) get their content from our knowledge of how those looks will vary with motion and other conditions. But even considered as a story about content fixation, important challenges remain, as we shall now see. 2. Constitutive Force? A specific dance, at some appropriate level of description, might plausibly be identified with a specific pattern of bodily motions. It may make no sense to suppose that one could know the dance without knowing something of those specific patterns of bodily motion. In this way, we espy something like a conceptual link between the dance and the details of it's embodied realization. At first blush, however, the link between our sensorimotor knowledge and skills and the contents of our perceptual experience looks less direct, as if the sensorimotor routines might be the hooks that reel in the contents, but in a merely causal fashion. The very same contents, one might well suspect, could be present in systems whose sensorimotor routines were very different to our own, or perhaps even in systems that were sensorimotor inert. I take it that these are the sorts of static internalist intuition that Noë (admirably) wants to unseat. One way to unseat them would be to embrace a radical option that Noë (p.89 and elsewhere) rejects, describing it as unacceptably behaviorist. The option would be to depict at least some basic forms of perceptual content as necessarily involving dispositions to act in certain ways (see Evans (1985) for a classic version of such a line). This seems plausible for e.g. certain egocentrically defined contents, such as that of a sound's appearing to come from over there. Entertaining such a content may in part consist in a swathe of dispositions to orient towards the sound. Subtract those dispositions to act and you must (if such an account is right) subtract the perceptual content itself. Noë depicts his own view in contrast to this, and as involving, in the case of a visually perceived flicker on the right, only 'practical knowledge of how movement would bring the thing into view' (89). By moving us squarely back into the realm of knowledge, however, Noë runs a risk of letting internalism in through the back door, and creates an internal tension between two components of his own account. One component is the oft-repeated idea that what counts are our expectations (or our implicit knowledge) concerning the way sensory stimulation will unfold. The other component is the idea that we try to bridge the explanatory gap (between physical goings-on and conscious experience) "by expanding our conception of the substrate in terms of which we hope to explain consciousness" (226). This expansion looks to the way neural activity supports embodied action as the missing link in explanations of perceptual consciousness. Here, the general claim is that "what determines phenomenology is not neural activity set up by stimulation as such, but the way the neural activity is embedded in a sensorimotor dynamic"